A graphic user interface (GUI), typically includes a collection of objects, such as screen controls, that are displayed on a display screen of a computer. The correct interpretation of the displayed objects is generally apparent to a properly trained human operator who interacts with the GUI.
For the purpose of some applications, it is necessary that the displayed objects of the GUI be automatically interpreted. Examples of such applications include automatic software testing, where an automatic software testing application may automatically interact with the GUI. Other examples include user monitoring (where an interaction of the user with the GUI is interpreted), speech activation devices, automatic generation of user interfaces for disabled people, and GUI design. Such an application may be required, for example, to automatically identify a screen control of the GUI without access to an object model of the screen controls.
Similar operations may be required in interpreting and extracting information from a visually structured document, such as a Portable Document Format (PDF) document.